Search for: All records

Generative AI holds promise for advancing human factors and ergonomics. However, limited training data can reduce variability in generative models. This study investigates using transfer learning to enhance variability in generative posture prediction. We pre-trained a conditional diffusion model on lifting postures where hands are near body center and fine-tuned it on limited extended-reach postures. Compared to training from scratch, transfer learning significantly improved joint angle variability across multiple body segments while maintaining similar accuracy in posture similarity and validity. Additionally, it reduced training time by over 90%, demonstrating efficiency benefits. These findings highlight transfer learning’s potential to enrich generative model outputs with more variable ergonomics data, supporting scalable and adaptive workplace design. 

Workers performing repetitive lifting tasks are at high risk of developing low-back work-related musculoskeletal disorders. While the Revised NIOSH Lifting Equation (RNLE) is a widely used tool for evaluating lifting-related risks, its reliance on manual measurement limits its scalability and efficiency. This study proposes a computer vision-based framework that automates RNLE computation using video data. The method integrates three key stages: (1) pose estimation to extract 3D joint coordinates, (2) lifting action recognition via kinematic features and a k-TSP classifier, and (3) estimation of RNLE multipliers from joint data. Applied to 40 lifting trials with motion capture-based ground-truth, the system achieved a coefficient of determination of 0.82 and a mean absolute error of 2.72 kg in estimating recommended weight limits. These findings demonstrate the potential of computer vision to automate ergonomic risk assessments. Future work will aim to expand task diversity and integrate coupling assessment for full RNLE coverage. 

ObjectiveTo identify lifting actions and count the number of lifts performed in videos based on robust class prediction and a streamlined process for reliable real-time monitoring of lifting tasks. BackgroundTraditional methods for recognizing lifting actions often rely on deep learning classifiers applied to human motion data collected from wearable sensors. Despite their high performance, these methods can be difficult to implement on systems with limited hardware resources. MethodThe proposed method follows a five-stage process: (1) BlazePose, a real-time pose estimation model, detects key joints of the human body. (2) These joints are preprocessed by smoothing, centering, and scaling techniques. (3) Kinematic features are extracted from the preprocessed joints. (4) Video frames are classified as lifting or nonlifting using rank-altered kinematic feature pairs. (5) A lifting counting algorithm counts the number of lifts based on the class predictions. ResultsNine rank-altered kinematic feature pairs are identified as key pairs. These pairs were used to construct an ensemble classifier, which achieved 0.89 or above in classification metrics, including accuracy, precision, recall, and F1 score. This classifier showed an accuracy of 0.90 in lifting counting and a latency of 0.06 ms, which is at least 12.5 times faster than baseline classifiers. ConclusionThis study demonstrates that computer vision-based kinematic features could be adopted to effectively and efficiently recognize lifting actions. ApplicationThe proposed method could be deployed on various platforms, including mobile devices and embedded systems, to monitor lifting tasks in real-time for the proactive prevention of work-related low-back injuries. 

This study presents a mobile app that facilitates undergraduate students to learn data science through their own full body motions. Leveraging the built-in camera of a mobile device, the proposed app captures the user and feeds their images into an open-source computer-vision algorithm that localizes the key joint points of human body. As students can participate in the entire data collection process, the obtained motion data is context-rich and personally relevant to them. The app utilizes the collected motion data to explain various concepts and methods in data science under the context of human movements. The app also visualizes the geometric interpretation of data through various visual aids, such as interactive graphs and figures. In this study, we use principal component analysis, a commonly used dimensionality reduction method, as an example to demonstrate the proposed learning framework. Strategies to encompass other learning modules are also discussed for further improvement. 

In recent years, there has been a trend to adopt human-robot collaboration (HRC) in the industry. In previous studies, computer vision-aided human pose reconstruction is applied to find the optimal position of point of operation in HRC that can reduce workers’ musculoskeletal disorder (MSD) risks due to awkward working postures. However, the reconstruction of human pose through computer-vision may fail due to the complexity of the workplace environment. In this study, we propose a data-driven method for optimizing the position of point of operation during HRC. A conditional variational auto-encoder (cVAE) model-based approach is adopted, which includes three steps. First, a cVAE model was trained using an open-access multimodal human posture dataset. After training, this model can output a simulated worker posture of which the hand position can reach a given position of point of operation. Next, an awkward posture score is calculated to evaluate MSD risks associated with the generated postures with a variety of positions of point of operation. The position of point of operation that is associated with a minimum awkward posture score is then selected for an HRC task. An experiment was conducted to validate the effectiveness of this method. According to the findings, the proposed method produced a point of operation position that was similar to the one chosen by participants through subjective selection, with an average difference of 4.5 cm. 

Work-related musculoskeletal disorders (MSDs) are often observed in human-robot collaboration (HRC), a common work configuration in modern factories. In this study, we aim to reduce the risk of MSDs in HRC scenarios by developing a novel model-free reinforcement learning (RL) method to improve workers’ postures. Our approach follows two steps: first, we adopt a 3D human skeleton reconstruction method to calculate workers’ Rapid Upper Limb Assessment (RULA) scores; next, we devise an online gradient-based RL algorithm to dynamically improve the RULA score. Compared with previous model-based studies, the key appeals of the proposed RL algorithm are two-fold: (i) the model-free structure allows it to “learn” the optimal worker postures without need any specific biomechanical models of tasks or workers, and (ii) the data-driven nature makes it accustomed to arbitrary users by providing personalized work configurations. Results of our experiments confirm that the proposed method can significantly improve the workers’ postures. 

Human-robot collaboration (HRC) is an emerging research area that has gained tremendous attention from both academia and industry. Since some robot-related factors can elicit mental stress or have negative psychological effects on human workers, it is essential to understand these factors and maintain workers’ mental stress at a low level. Galvanic Skin Response (GSR) measures skin conductance and is known to be a physiological measurement that reflects short-term mental stress. Typically, skin conductance increases in response to greater mental stress. In this study, the mental stress caused by the hand-over activities of a collaborative robot was investigated using both GSR as an objective measurement and NASA-Task Load Index (TLX) as a subjective assessment. Several robot-related factors that may lead to mental stress were experimentally examined. GSR outcomes indicated that end effector approaching within workers’ view, low end effector speed, and constrained end effector trajectory led to a significantly lower skin conductance. Some aspects of the NASA-TLX also indicated that speed and trajectory significantly affected the scores. Yet, no significant differences were found between approaching directions regarding NASA-TLX scores. 

ObjectiveThis study aims to improve workers’ postures and thus reduce the risk of musculoskeletal disorders in human-robot collaboration by developing a novel model-free reinforcement learning method. BackgroundHuman-robot collaboration has been a flourishing work configuration in recent years. Yet, it could lead to work-related musculoskeletal disorders if the collaborative tasks result in awkward postures for workers. MethodsThe proposed approach follows two steps: first, a 3D human skeleton reconstruction method was adopted to calculate workers’ continuous awkward posture (CAP) score; second, an online gradient-based reinforcement learning algorithm was designed to dynamically improve workers’ CAP score by adjusting the positions and orientations of the robot end effector. ResultsIn an empirical experiment, the proposed approach can significantly improve the CAP scores of the participants during a human-robot collaboration task when compared with the scenarios where robot and participants worked together at a fixed position or at the individual elbow height. The questionnaire outcomes also showed that the working posture resulted from the proposed approach was preferred by the participants. ConclusionThe proposed model-free reinforcement learning method can learn the optimal worker postures without the need for specific biomechanical models. The data-driven nature of this method can make it adaptive to provide personalized optimal work posture. ApplicationThe proposed method can be applied to improve the occupational safety in robot-implemented factories. Specifically, the personalized robot working positions and orientations can proactively reduce exposure to awkward postures that increase the risk of musculoskeletal disorders. The algorithm can also reactively protect workers by reducing the workload in specific joints.